145 research outputs found
VarArray Meets t-SOT: Advancing the State of the Art of Streaming Distant Conversational Speech Recognition
This paper presents a novel streaming automatic speech recognition (ASR)
framework for multi-talker overlapping speech captured by a distant microphone
array with an arbitrary geometry. Our framework, named t-SOT-VA, capitalizes on
independently developed two recent technologies; array-geometry-agnostic
continuous speech separation, or VarArray, and streaming multi-talker ASR based
on token-level serialized output training (t-SOT). To combine the best of both
technologies, we newly design a t-SOT-based ASR model that generates a
serialized multi-talker transcription based on two separated speech signals
from VarArray. We also propose a pre-training scheme for such an ASR model
where we simulate VarArray's output signals based on monaural single-talker ASR
training data. Conversation transcription experiments using the AMI meeting
corpus show that the system based on the proposed framework significantly
outperforms conventional ones. Our system achieves the state-of-the-art word
error rates of 13.7% and 15.5% for the AMI development and evaluation sets,
respectively, in the multiple-distant-microphone setting while retaining the
streaming inference capability.Comment: 6 pages, 2 figure, 3 tables, v2: Appendix A has been adde
Profile-Error-Tolerant Target-Speaker Voice Activity Detection
Target-Speaker Voice Activity Detection (TS-VAD) utilizes a set of speaker
profiles alongside an input audio signal to perform speaker diarization. While
its superiority over conventional methods has been demonstrated, the method can
suffer from errors in speaker profiles, as those profiles are typically
obtained by running a traditional clustering-based diarization method over the
input signal. This paper proposes an extension to TS-VAD, called
Profile-Error-Tolerant TS-VAD (PET-TSVAD), which is robust to such speaker
profile errors. This is achieved by employing transformer-based TS-VAD that can
handle a variable number of speakers and further introducing a set of
additional pseudo-speaker profiles to handle speakers undetected during the
first pass diarization. During training, we use speaker profiles estimated by
multiple different clustering algorithms to reduce the mismatch between the
training and testing conditions regarding speaker profiles. Experimental
results show that PET-TSVAD consistently outperforms the existing TS-VAD method
on both the VoxConverse and DIHARD-I datasets.Comment: Submission for ICASSP 202
- …